CONVERGENCE OF EXPECTED UTILITIES WITH 
ALGORITHMIC PROBABILITY DISTRIBUTIONS 



PETER DE BLANC 
DEPARTMENT OF MATHEMATICS 
O ' TEMPLE UNIVERSITY 

O 

<N 
O 

Q 

oo 

(N 



< 

o 



> 

00 

m 

(N 

o 



- 1—^ 

x 



Date: February 2, 2008. 

Thanks to Nick Hay at Cornell University for help with the Recursion Theorem. 





convergence of expected utilities with algorithmic probability distributions 

1. Abstract 

We consider an agent interacting with an unknown environment. The environ- 
ment is a function which maps natural numbers to natural numbers; the agent's 
set of hypotheses about the environment contains all such functions which are com- 
putable and compatible with a finite set of known input-output pairs, and the agent 
assigns a positive probability to each such hypothesis (Probability distributions over 
all computable functions are used in theoretical AI systems such as AIXI (Hutter, 
2007)). We do not require that this probability distribution be computable, but it 
must be bounded below by a positive computable function. 

The agent has a utility function on outputs from the environment. We show 
that if this utility function is bounded below in absolute value by an unbounded 
computable function, then the expected utility of any input is undefined. 

This implies that a computable utility function will have convergent expected 
utilities iff that function is bounded. 

2. Notation 

Here we set up our notation. 

Let R be the set of partial /i-recursive functions, and let S = R fl N N ; that is, 
S is the set of total \x- recursive functions on a single argument. We will use an 
index R — {4> n }^Loi where our index is a Godel numbering of the computable 
functions. 

Let h G N N be the true function that describes the environment. The agent has 
some finite set of tested inputs /, so the agent knows the value of h(i) for all i G I. 
Let Si = {/ € S : (Vi G /), f(i) = h(i)}. That is, Sj is the set of hypotheses which 
agree with the agent's knowledge of tested inputs. 

Let H be our set of hypotheses about the environment, with Si C H C N N . 

Let p : H — > M be our probability distribution on the hypothesis set. We require 
that there exist some computable function p : N — > Q such that (V^)„ G 57), < 
p(n) <p((f> n ). 

Let U : N — > K be the agent's utility function. We suppose that U is unbounded, 
and that there exists an unbounded computable function U : N — > Q : (Vn G 
N),\U(n)\<\U(n)\. 

3. Proof of Divergence of Expected Utilities 

Fix fee (N — I), an input whose expected utility we will consider. Then define 
B : N -> N, with 

(1) (Vx G N),B(x) = max{0„(fc) : n G N, n < x} 

The sequence {£?(j)}°^l can be thought of as an analog of the Busy Beaver 
numbers (Rado, 1962). This sequence will be used in proving our main result. 
Note that B is not a computable function, as we are about to show. 

Lemma 1. Let f G S. Then B{x) > f(x) infinitely often. 

Proof. Suppose not. Then B{x) > f(x) only finitely many times, so let F(x) = 
1 + f(x) + max {5(2;) - f(x) : x G N}. Then F G 5, and (Vx G N), F(x) > B(x). 

Let Q : N 2 — > N, Q(i, x) = F(i). By a corollary of the recursion theorem (Kleene, 
1938), there exists p G N such that (Vx G N), <j> p (x) = Q(p, x) = F(p). 
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By equation [U B(p) > <p p {k) = F{p). This contradicts our statement that 
(Vx G N),F(x) > B[x). □ 

The expected utility of inputting k to the environment is: 

(2) EU(h(k))=J2p(f)U(f(k)) 
Theorem 1. The series in equation^ does not converge. 

Proof. To establish that this series does not converge, we will show that infinitely 
many of its terms have absolute value > 1. We will do this by constructing a 
sequence of hypotheses in Si whose utility grows very quickly - as quickly as the 
function B - faster than their probabilities can shrink. 

By equation [TJ for each j G N there exists uj e N, Uj < j such that (f> Uj (k) = 
B(j). Now we define a map on function indices G : N — > N such that: 

. , _ J h{x) if x 6 I 

<PG(n)W - | min {y£N: \U(y)\ > \4> n (k)\} otherwise 

Because \U\ is an unbounded function, <pG(n) is a total function as long as 4>n{k) is 
defined. By definition, (Vn <E N), 4>G(n) E Si C H. 

So our sequence of hypotheses will be {<f>G(u j )}'*L 1 - Because k £ I, we have: 

(3) (Vj g N), |C/(^ G( ».)(fc))| > |C/(0 G(Uj) (fc))| > <t> Uj {h) = B(j) 
We're almost done. We now let q : N — > N, with: 



(4) <z(x) = Tsup { -^ : y £ N, y < x}] 

Then q is a nondecreasing function in S* with the property that (Vx S N), q(x) > 
p(G(x)) _1 . By Lemma[l] B(x) > q(x) infinitely often. It follows from equations^] 
and H] that: 

(5) \Uy GM (k))\ > B(j) > q(j) > ^^(Ui))- 1 ^p^o^))- 1 

for infinitely many j G N. 

Because p(G(uj)) is always positive, and by the above equation, it follows that: 

(6) |p(0G(n 3 ))C/(^ G( «,)(fc))l>l 

infinitely often. Because these are all terms of the series in equation [21 the series 
can not converge. □ 



4. Remarks 

We have shown that utility functions which are bounded below in absolute value 
by an unbounded computable function do not have convergent expected utilities. 

Because all computable functions are bounded below in absolute value by them- 
selves, it follows that all unbounded computable utility functions have divergent 
expected utilities. Since bounded utility functions always have convergent expected 
utilities, we know that a computable utility function has convergent expected utilities 
iff it is bounded. 
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